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(54) DISPLAY METHOD 

(57)Abstract: 

PROBLEM TO BE SOLVED: To make communication, 
rich in presence by transmitting a sentence having 
expressions which are intended by a transmitter. 
SOLUTION: This display method comprises a step for 
detecting an emotion indicating symbol included in a 
text, a step for extracting the animation data of a facial 
image corresponding to the detected emotion noticing 
symbol, and a step for displaying the animation data, at a 
position where the emotion noticing symbol is detected, 
while displaying the text. 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1]The method of presentation comprising: 

A step which detects a feeling notation sign contained in a text. 

A step which extracts animation data of a face picture corresponding to said detected feeling 
notation sign based on a correspondence table which matches said feeling notation sign and 
animation of said face picture, A step which displays said animation data in a position from which 
said feeling notation sign was detected while displaying said text. 



[Translation done.] 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention]In a cellular phone, a portable remote terminal, a personal computer, etc., 
this invention displays text data and face image information, such as an E-mail, and relates to 
the method of presentation for changing a face picture according to the contents of text data. 
[0002] 

[Description of the Prior Art]since create bitter feelings in the transfer only by text data, it is 
only taken by the different meaning or there is nothing, when transmitting information with an E- 
mail etc. conventionally — ( — ~\) — the text was created, weaving the feeling notation sign 
[ like ] called what is called a face mark into the text. 

[0003] By the way, since an intention can be appropriately transmitted compared with the case 
where text data are transmitted if picture information is used when transmitting information, it is 
possible to use a TV phone etc. However, the fault that picture information is too large and a 
picture becomes breaks off and breaks off without transmission going smoothly can be 
considered. It is actual that the face of the person himself/herself as it is has not spread for the 
reason the inside of the room in which it is reflected with reality will become open. 
[0004]The problem that the picture information mentioned above is large, and the problem only 
in text data that transfer of an intention is difficult are received, The text display of a text and 
read-aloud of a text were performed, and there were some which display the animation made to 
change the opening part of a face picture as if it had talked along with the text to the face 
picture at the time of usual [ of delivery model people ], or the face picture of a character. 
However, even if it was this method, the image data which carried out the form of the mouth for 
every single character was saved, or it was required to change a picture. 

[0005]By the way, there is not only modification of such a mouth but a thing which can transmit 
an informer's intention further by changing and displaying expression, such as a smiling face and 
a sad face. For example, the mode of expression of texts arbitrary to JP,1 1-242751, A and a jam, 
"surely, The mood information which the former calls "hearsay" and the latter calls "firm belief" 
from that it is — " and the text "being based on — is — that's right" was pulled out, and the 
animation display was controlled by the expression data corresponding to this mood information. 
According to JP,6-162167,A, it is a display side, it is a creation side of text data so that a 
display as the person of the creation side of the text meant may be attained in displaying 
synthesized speech or face image composing based on text data, and the parameter for 
expression generation is created, and the display side is passed. 

[0006]The text voice conversion device which reads out an e-mail address, URL, a face mark, 

etc. appropriately by synthesized speech is indicated by JP,1 1-305987.A. 

[0007] 

[Problem(s) to be Solved by the Invention]However, in JP,1 1-242751, A mentioned above. In 
order to have to consider it as a text of a fixed form which corresponds to mood information 
when inputting a text since a system recognizes the mood information of hearsay or firm belief 
automatically. There was a problem that a possibility that the state of mind which is not adapted 
for the individual text for which a comparatively soft expression is used easily, and a user means 
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cannot be expressed was high. 

[0008]In JP,6-162164 f A, it was a creation side, and since it was necessary to create the 
parameter for expression generation, there was a problem that the work of what can express a 
state of mind correctly will become complicated. 

[0009]In JP.1 1-305987 A since there was nothing against which it appeals to vision only as 
reading out a face mark as a sound, and there is a problem that an impression will become thin 
and the sound of the face mark was read out in the insertion point of the face mark, there was a 
problem that the contents of the text will break off. 

[0010]Then, according to the method of presentation concerning this invention, a text is only 
created in the way which writes the usual E-mail etc., and it aims at the ability to transmit 
expression as the transmitting side meant to a recipient. 
[0011] 

[Means for Solving the Problem]A step which detects a feeling notation sign contained in a text 
in order to solve a technical problem mentioned above, A step which extracts animation data of 
a face picture corresponding to said detected feeling notation sign based on a correspondence 
table which matches said feeling notation sign and animation of said face picture, The method of 
presentation which consists of a step which displays said animation data in a position from which 
said feeling notation sign was detected is provided displaying said text. 
[0012] 

[Embodiment of the Invention] Hereafter, the method of presentation concerning this invention is 
explained with reference to drawings. Dra wing 1 is the E-mail transmission and reception system 
which applied the method of presentation of this invention, and dra w ing 2 is a flow chart which 
shows the operation. 

[0013]The transmission and reception means 2 transmits and receives the message of an E-mail 
via the antenna 1. The communication control means 3 decoded the sound inputted from the 
signal inputted from the transmission and reception means 2 t or the microphone 5, and has 
encoded the sound outputted from the signal outputted from the transmission and reception 
means 2, or the loudspeaker 4. The central controlling means 6 is the so-called CPU portion of a 
personal computer, and is connected to the communication control means 3, the memory buffer 
7, the message preparing means 8, and mutual. It is connected also with the character color 
alteration means 10. The memory buffer 7 accumulates an incoming message and an attached 
file, when e-mail is received, and when transmitting e-mail, it accumulates an outgoing message 
and the attached file for transmission. 

[0014]By the message preparing means 8, the keyboard etc. which are not illustrated are used, 
and a text is inputted, using the feeling notation sign 9. This is performed in the same way as 
inputting the text of an E-mail, weaving in the usual face mark. The feeling notation sign 9 may 
also contain what put a Chinese character single character showing expression other than a face 
mark as shown in d rawing 3 in the parenthesis. These feeling notation signs 9 may express the 
same feeling, and may change the degree of expression by the notation. "-" expresses the 
laughter of a grade which smiles and, specifically, "-" expresses great laughter. 
[0015]And after ending the input of a text, the character color alteration means 10 operates and 
it is discolored at the speed of a text which reads a text exactly flatly. It is confirmed 
simultaneously whether the search means 1 1 is the feeling notation sign 9 which the discolored 
character mentioned above. If the feeling notation sign 9 is discovered in a text, the same sign is 
discovered from the animation correspondence table 12, and corresponding animation is 
specified. At this time, when the feeling notation sign 9 is it is the animation of the face 
which goes by steps ******, and when corresponding animation is [ the feeling notation sign 9 ] 

-", after smiling, it is animation which changes to great laughter. 
[0016]And if the animation data number corresponding to the feeling notation sign 9 is gained, a 
data number will be sent to the reproduction means 13. In the reproduction means 13, animation 
data is chosen from the data storage part 14, and animation is reproduced and displayed. Thus, 
the discoloration and animation reproduction of a character color are repeated until the input of 
a text is completed. 

[0017]Next, operation of the method of presentation which starts this invention using dra wing 2 
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is explained. In this example, the example "hello, it is [ !-today ] cold" (TT) is used. First, the 
input of the duty is performed by the sending person (S1). Since the cursor for an input is 
coming to the sentence end of the text by the stage which the input ended, it returns to the 
beginning of a sentence of the text (S2). Next, it changes and goes by the same way as the 
discoloration of words [ in / for the character color of the duty / karaoke ] (S3), the red 
character as that to which red changes color of a black character string in the case of the 
example of this example — "**" ->"it does not come" -> — "— it does not come — alike — "- 
- as — it increases and goes. And whenever the character string of discoloration increases, the 
character string is a feeling notation sign, or it searches using an animation correspondence 
table (S4). 

[001 8]The example of search is shown below, the face mark which is a feeling notation sign — 
usually — " (" — since it starts — " (" — if detected, comparison with a feeling notation sign 
will be started.)) If a single-character eye agrees, and the following character will be compared 
and it will agree to the last "", it will judge with it being the feeling notation sign which agreed. 
[0019]When "-" which is a feeling notation sign is found by such search (it is Y at S4), the 
animation data number 3 which progresses to S5 and corresponds is gained (S5). On the other 
hand, when it is not a feeling notation sign (it is N at S4), it returns to S3 and change of a 
character color is continued. 

[0020]If the animation data number 3 can be gained in S5, the animation data corresponding to 
the animation data number 3 will be read and displayed from the data storage part 14 (S6). Then, 
since "-" of the animation data number 3 is animation which changes to great laughter after 
smiling, it displays the animation. 

[0021 ]And since it investigates whether it is this end (S7) and the text has not been completed 
yet, it returns to S3 and a character color is changed. Then, since "" (TT) which is a feeling 
notation sign was found, the animation data number 6 can be gained and the display of the 
animation data number 6 is performed. In this example, since the text is ended here (it is Y at 
S7), processing is ended. 

[0022]Next, transmitting and receiving processing is explained. In the case of transceiving 
equipment provided with the method of presentation of this invention, the information which 
should be transmitted is only a mail header and the text as well as a general E-mail. Having used 
the method of presentation of this invention for the mail header is indicated, and when the 
receiver has not adopted the method of presentation of this invention, it becomes the display of 
a general E-mail. On the other hand, when the receiver has also adopted the method of 
presentation of this invention, the central controlling means 6 displays the contents of the mail 
accumulated in the memory buffer in the following way. 

[0023]The character color alteration means 10 starts change of a character color from the head 
of the text. One character after another is changed into red as mentioned above. And it 
searches with the search means 1 1 whether the character which had the character color 
changed is a feeling notation sign, and when it is a feeling notation sign, the animation data 
number 12 is gained from an animation correspondence table, the data is reproduced by the 
reproduction means 13, and it displays on the displaying means 15. The animation data and the 
animation correspondence table of this example are common to the transmitting side and a 
receiver. 

[0024]As mentioned above, only by describing the E-mail which uses the face mark which has 
usually transmitted, since the animation correspondence table and the animation data storage 
part are beforehand provided in the receiver, The face picture of the animation which attached 
an expression intended to the timing which the sending person meant can be displayed, and the 
contents of the E-mail are transmitted more to a partner with one-layer presence. Since an E- 
mail can be read looking at an animation picture also by a receiver, an E-mail can be received 
much more happily. 

[0025]Next, the 2nd example of the method of presentation concerning this invention is 
described. Di^jng_5js the E-mail transmission and reception system which applied the method 
of presentation concerning this invention, and drawing 6 is a flow chart which shows the 
operation. About the same component as the E-mail transmission and reception system of 
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drawin g^, explanation is omitted in part. According to drawing 2 . the message preparing means 8, 
the character color alteration means 10, the face data setting-out means 21, and the editing 
means 22 besides the memory buffer 7 are connected to the central controlling means 6. Here, if 
face data points out the data file in which vector coordinates of a face part article, a color of 
each face part article, etc. which constitute a face picture were described and a face part article 
is plotted as coordinates, the face picture of an expressionless state of rest will be displayed. 
The editing means 22 can edit some face data, or it can change the degree of the expression of 
a face picture so that it may mention later. 

[0026]The changed data is overwritten by the preserving means 23 to the face data storage 
parts store 24 and the expression table 25. At this example, by reading and performing 
expression operation data (function etc.) to an expressionless face picture, expression of a face 
picture can be made into a smiling face, or can be made into a tearful face. An expression table 
is shown in dj-a_wing_7_and is making the degree of the expression of expression operation data, 
and a laughter face and a tearful face correspond to a feeling notation sign at this time. 
[0027]Next, the flow of processing is explained using drawing 6 . If creation of a message is 
started, a sending person will input the text using a feeling notation sign (S1 1). next, a sending 
person — the face data of the person himself/herself is set up (S12). At this time, a face picture 
may be simultaneously drawn by the displaying means 15. Next, if the depression of the 
confirmation switch formed in the message preparing means 8 that a sending person should 
check expression is carried out (it is Y at S13), the character color alteration means 10 changes 
the character color into red from the character of the beginning of a sentence of the text (S14). 
When the search means 1 1 judges that the character is a feeling notation sign, in (S15 And Y), 
The grade (value) of an expression corresponding from the expression table 25 is acquired (SI 6), 
and the face data set up previously is read from the face data storage parts store 24 and the 
expression operation data storage parts store 26, and the reproduction means 13 is performed 
until the grade of expression becomes previous ** (S17). 

[0028]Expression operation data "laugh" is a public function of a Face class, and, specifically, is 
a laugh method (function) which has an int type integer in an argument. When displaying a face, 
the object face of a Face class is generated. And among a text, if face mark "-" is found, 
face.laugh (10) will be called. It repeats 10 times that the contents of face.laugh perform the 
laugh method of a left eye (left eye), a right eye (right eye), the upper lip (upper lip), and a 
metastoma (lower lip), and draw. The laugh method of a Face class and the laugh method of a 
LeftEye class are shown in drawing 8. 

[0029]each coordinate value which constitutes a left eye whenever it draws according to the 
figure — every [ value ] — it increases and becomes the eye at which it laughed. A right eye 
(right eye.laugh), the upper lip (upper lip.laugh), and a metastoma (lower lip.laugh) are also the 
same. Thus, drawing is performed and the animation display of the face of assumption at which it 
laughs is carried out (S18). 

[0030]Here, it is assumed that it had shifted from the feeling which signs that the face picture 
laughed want to express on the face of the document. When such, it edits (it is Y at S19). Then, 
the editing means 22 newly displays a face picture and an editing window on the displaying 
means 15 so that the grade of the expression in the feeling notation sign can be changed, for 
example. And if the grade at which it laughs with a scroll bar is enlarged (S20) and set up, the 
value of the grade which has hung on the expression table 25 by the preserving means 23 will be 
updated to 20 (S21). It returns to the text and a character color is changed. By this repetition, 
the expression which the sending person meant can be set as each feeling notation sign. 
[0031]And if the discoloration of the text is completed (it is Y at S22), creation processing of a 
message will be ended. In the editing means 22, in addition to this, the color of a face part article 
can be set up, or a face picture outline can also be edited. In this case, the data of the face data 
storage parts store 24 will be updated. In the case of transmission, the face data and the 
expression table which were set up besides a mail header and the text are transmitted. 
[0032]When the e-mail transmission and reception system which applied the method of 
presentation which the receiver mentioned above is introduced, expression operation data can 
use a common thing. The expression to which the feeling notation sign was set by the way with 
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change of the character color is displayed by the same processing as the expression check of 
the transmitting side which the receiver also mentioned above then, and the contents of the E- 
mail are transmitted as a sending person's intention by it. 

[0033]As mentioned above, it is possible to express as the timing and expression which face 
data and an expression table other than an electronic mail body are only transmitted, and are a 
sending person's faces, and the sending person meant. In the second example, when it cannot be 
satisfied with the grade of a predetermined expression, expression can be edited easily, and 
since this changed part is a mere setting variation, it can be transmitted, without making the 
amount of information increase by in any way edit before. Although the state where expression 
operation data was beforehand memorized as an example at the e-mail transceiving equipment 
side was explained, of course, the operation objects (execution program) for expression 
operation may be created at the transmitting side, and it may be transmitted. In this case, it is 
not necessary to have expression operation data in a receiver, and even when the receiver has 
not introduced the e-mail transmission and reception system which applied this invention, it can 
display. 
[0034] 

[Effect of the Invention]As mentioned above, as explained in full detail, according to the E-mail 
transmission and reception system which applied the method of presentation concerning this 
invention, an E-mail with expression as the sending person meant can be transmitted in the way 
which describes the usual E-mail. It is effective in the ability to perform by this electronic mail 
communication which is full of presence. When expression of the face picture displayed cannot 
be satisfied with the expression decided beforehand, the grade of expression can be edited easily 
and it is effective in the ability to transmit without making the amount of information increase by 
edit before. Even when the E-mail transmission and reception system which applied the method 
of presentation where a receiver starts this invention temporarily when expression operation 
data is an object is not adopted, it is effective in the E-mail which is full of presence with 
expression being receivable. 



[Translation done.] 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

LThis document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawin g J]lt is a block diagram showing the composition of the E-mail transmission and 
reception system which applied the method of presentation concerning this invention. 
[ Drawin g_2]It is a figure showing one example of the method of presentation concerning this 
invention. 

[Drawing 3]lt is a figure showing an example of a feeling notation sign. 

[Drawin gillt is a figure showing the animation correspondence table used with the method of 
presentation concerning this invention. 

[Draym^Sllt is a figure showing other composition of the E-mail transmission and reception 
system which applied the method of presentation concerning this invention. 
[ Drawing 6]It is a figure showing the second example of the method of presentation concerning 
this invention. 

Jawing: 7]It is; a figure showing the expression table in the second example of the method of 
presentation concerning this invention. 

jDt^win^SlIt is a figure showing laugh in the second example of the method of presentation 
concerning this invention. 
[Description of Notations] 

1 Antenna 

2 Transmission and reception means 

3 Communication control means 

4 Loudspeaker 

5 Microphone 

6 Central controlling means 

7 Memory buffer 

8 Message preparing means 

9 Feeling notation sign 

10 Character color alteration means 

1 1 Search means 

12 Animation correspondence table 

13 Reproduction means 

14 Data storage part 

15 Displaying means 



[Translation done.] 
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[0 0 3 2] Sfll**±i6Lfc**36rifeSraffl Lfcy- 
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[08] 

class Face{ 

LeflEye left_eye; 
RightEye right_eye; 
Upper Lip upper Jip; 
Lowerlip lower Jip; 

public void laugh (int X){ 

foitfnt i=0; KX;i++){ 

left_eye.laugh(i); 

right__eyeJ.augh(i); 

upper Jip iaugh(i); 

lower _Hp.laugh(i); 

displayO; //SlKD^tR 

} 

} 

} 

class LeftEye{ 

int np = 8; ZMtitt SffifflkDft 

int pQ = new intfnp]; &t*£f ( X . y ) 8 tf-f > h 

(**) 

public void left_eye(mt time){ 

int value - 2; /«««©«MiaS 
for(int i=0; i<np; i++){ 

p[i] += value*tinie; 

* 



